NYNET Communication System (NCS): A Multithreaded Message Passing Tool over ATM Network

Current advances in processor technology, and the rapid development of high speed networking technology, such as ATM, have made high performance network computing an attractive computing environment for large-scale High Performance Distributed Computing (HPDC) applications. However, due to the communications overhead at the host-network interface, most of the HPDC applications are not getting the full beneet of high speed communication networks. This overhead can be attributed to the high cost of operating system calls, context switching, the use of ineecient communication protocols, and the coupling of data and control paths. In this paper, we present an architecture and implementation for a low-latency, high-throughput message passing tool, that we refer to as the NYNET (ATM wide area network testbed in New York state) Communication System (NCS), which can support a variety of HPDC applications with diierent Quality of Services (QOS) requirements. NCS uses multithreading to provide eecient techniques that overlap computation and communication. NCS uses read/write trap routines to bypass traditional operating system calls. This reduces latency and avoids using ineecient communication protocols. By separating data and control paths, NCS eliminates unnecessary control transfers. This optimizes the data path and improves the performance. Benchmarking results show that the performance of NCS is at least a factor of two better than the performance of corresponding p4 and PVM primitives.